1. Field of the Invention
The present invention relates to a technique for calibrating parameters associated with an image capturing apparatus.
2. Description of the Related Art
Since a lens of a camera which is generally used is not the one of an ideal pin-hole camera, an image captured by such camera often includes distortions such as a barrel distortion and the like caused by the lens, and it is not an ideal perspective projection. Hence, when image processing is executed by a computer or the like, processing for calibrating distortions of an image is prevalently executed.
Conventionally, for such processing, a method of capturing an image of a calibration pattern, and calculating distortions and perspective projection conversion parameters of a camera from the image is used. S. Uchiyama, K. Takemoto, K. Satoh, H. Yamamoto, and H. Tamura: “MR Platform: A basic body on which mixed reality applications are built,” Proc. IEEE/ACM Int'l Symp. on Mixed and Augmented Reality (ISMAR 2002), pp. 246-253, 2002 describes such conventional image distortion calibration method.
For example, the following equations are known as a distortion model:x′=k2×(xi−cx)y′=k2×(yi−cy)d=1−k1(x′2+y′2)xo=x′×d+cx yo=y′×d+cy  (1)where (xi, yi) represents an ideal position free from any distortion, (xo, yo) represents a distorted position, and (cx, cy) represents the distortion central position.
A basic sequence for estimating parameters is as follows.
A) Initial values (cx, cy, k1, and k2) are prepared.
B) From grid-like observation points (xo, yo), points (xc, yc) after distortion correction are calculated using distortion parameters of process A).
C) A homography used to convert the grid points to (xc, yc) is calculated by the least square method.
D) (xh, yh) required to convert the grid points by the homography of process C) is calculated.
Note that processes C) and D) correspond to a manipulation for linearly approximating (xc, yc) in which the distortion remains.
E) (xh, yh) is inversely corrected by the distortion parameters of process A) to calculate (xho, yho).
F) Using the differences between (xo, yo) and (xho, yho), the distortion parameters are optimized by the Newton method.
However, in order to calculate the camera distortion as described above, the precondition for, e.g., capturing an image of known markers which are distributed on a plane perpendicular to the optical axis of the camera is required, and when the calibration pattern is placed not perpendicularly, errors occur.
Since the perspective projection conversion parameters of the camera must be separately calculated later, the calibration must be done twice to calculate the distortion parameters and perspective projection conversion parameters. For the second calibration, it is desirable to capture not only an image of the perpendicular calibration pattern but also images of the calibration pattern of a plurality of orientations, resulting in troublesome operations.
FIG. 3 is a flowchart of the aforementioned conventional distortion parameter calculation processing.
An actually captured image is acquired by capturing an image of a calibration pattern (step S301). Indices (observation points) in the acquired captured image are detected and their positions are acquired (step S302). Next, calibration parameters (distortion parameters) are calculated by the above-mentioned method (step S303). The calculated calibration parameters are saved (step S304).
Furthermore, after the processing according to the flowchart in FIG. 3, the perspective projection parameters of the camera must be calculated.
The above example does not consider a case wherein an aspect ratio includes errors. When the aspect ratio is not accurately 1:1, distortion parameters include errors.
As another prior art, Microsoft technical report: Technical Report MSR-TR-98-71 is available. This reference discloses a method of simultaneously calculating the perspective projection parameters of the camera and the distortion parameters. However, since the aspect ratio is not considered in this reference, errors of the aspect ratio cause those of the perspective projection parameters of the camera and the distortion parameters.